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1. INTRODUCTION 


1.1 Summary 


This report presents a new system for interactive computer aiding 
of group decision making, developed under the sponsorship of ARPA's Cybernetics 
Technology office. The group decision aid is supported by a PDP 11/45 
minicomputer using the UNIX operating system. It features simple individual 
data entry terminals and a large-screen color video display for feedback of 
computer-generated information. Its purpose is to guide the group decision 
making process by selective elicitation of a decision tree which incorporates 
value and probability inputs from all group members. A specially-trained 
system operator, called an intermediator, facilitates group interaction with 
the aiding program, so that group members need have no prior familiarity 
with computers or decision analysis. Initial trials with the aiding system 
were conducted uSing a crisis scenario formulated under subcontract by CACI, 
Washington, D.C. These trials were highly successful. Experimental groups 
were immediately able to use the system to generate decision trees of more 
than 40 nodes during three to four-hour discussion periods. Group members 
reported that the aid significantly enhanced the decision making process. 
The present report describes the current aiding system, and discusses its 
future development and application. 


1.2 Decision Analytical Approach 


Constant advancements in technology and communications, as well as 
the growing complexity of world affairs, have increased the importance of the 
decision making function. In today's military environment most upper-level 
decisions are made by committees and staff groups. In non-military environments 
as well, including those of government, business, medicine, and the law, 
group decisions often have a wide and substantial impact on future events and 
relationships. Decision making groups typically contain experts from several 
specialty areas, who bring to the decision environment disparate sets of values. 
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Decision time is usually limited, the decision making procedure is relatively 
unstructured, and intra-group conflicts arise on a broad variety of issues. 
The group usually cannot consider the maximum set of alternatives, conflicts 
tend to be resolved in an arbitrary manner, if at all, and as a consequence 
the resultant decision is rarely up to the aggregate potential of the group 
membership. 


Decision analysis offers a promising approach to solving these 
problems. The analytical procedure of building a decision tree formalizes _ 
the decision process, and permits incorporation of individual values (utilities) 
into the selection of alternative courses of action (Hays, O'Connor, Peterson, 
1975). However, decision analysis as it is usually practiced is a highly 
personal and time-consuming process. Trained decision analysts are generally 
used to assist in the solution of problems ranging over a large variety of 
domains. In most cases the decision analysts know far less about the problem- 
domain than do their clients. Joint education of the analysts and the clients 
takes up a disproportionate amount of decision making time. 


Accordingly, it appears highly worthwhile to automate the analytical 
process, using a domain-independent system to interrogate decision makers 
directly, and to construct a decision tree based on their responses. Leal 
and Pearl (1976) have shown that automated tree elicitation from individuals 
is feasible, and that on-line sensitivity analysis can be used to concentrate 
tree development on the branches with highest pay-off, thus streamlining the 
entire decision analysis process. Likewise, Gardiner and Edwards (1975) 
and Sheridan (1975) have shown that direct, real-time feedback of responses 
in group decision making focuses the effort on areas of real difference, 
while maintaining the advantages of full group participation. Finally, 
Decision and Designs, Inc. has shown the usefulness of automated decision 
aiding with a portable computerized system for choosing an optimum alternative 
given a wide range of expected event outcomes (Kelly and Stewart, 1977). 


The Perceptronics Group Decision Aid puts this aggregate decision technology 
at the command of a group of expert decision makers who must arrive at a 
decision solution in a relatively short amount of time. Focus is on problem 
structuring, on quick resolution of group conflicts, and on constant guidance 
on the most critical issues. 


One of the objectives of decision analysis is to provide simple formal 
procedures and structures for coding intuitive judgments about a particular 
problem situation. The types of problems most often treated by decision 
analytical methods are those in which the goal is to select one specific 
course Of action from among a number of contending alternative courses of 
action. This type of selection is defined as a "decision". The role of 
decision analysis is to provide a procedure for the elicitation of relevant 
problem information so that a "best" decision can be reached within reasonable 
amount of time. 


The approach taken by decision analysis assumes that people are able 
reliably to detect, to store, and to retrieve fragments of knowledge and 
information, but cannot as reliably aggregate these fragments into a global 
judgment. By forcing decision makers to decompose their decision problems 
into relevant components, judgmental values assigned to these components 
can be aggregated mathematically into a global recommendation. If the value 
estimates on each problem component can be individually justified, and if the 
aggregation method is sound, then the resultant global inference is both 
rational and acceptable. 


Decision Tree. One of the most common structures used in decision 
analysis is the decision tree. The decision tree permits the logical 
enumeration of possible decision actions and relevant event outcomes in the 
future. Potential future situations that could result as a consequence of 
the currently available action alternatives are analyzed and evaluated 
separately. Then, a formal algorithm can integrate these evaluations and 
exhibit their impact on the overall decision problem. 
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A simplified decision tree is shown in Figure 1-1. It begins with 
a listing of the major available alternatives, from which one must be selected. 
The square box is called a "decision node", and indicates that the branches 
emanating from it are to be considered as possible actions. The decision 
maker 1s free to choose one and only one of these actions. It is thus a 
necessary requirement of decision trees that all action lists must be mutually 
exclusive. Actions can precipitate other actions, and also events over which 
the decision maker has no control. Event nodes, shown as circles, have as 
branches all the outcomes that may occur at that point in the tree. — 


As the decision tree is expanded, more and more possible events and 
opportunities for actions arise. Thus, the tree looks forward into time. 
If one path from the beginning of the tree is followed to the end, it describes 
a possible future "scenario". At every decision node, the decision maker is 
free to choose one of the available alternatives; at every event node, an 
outcome will happen which is not under the decision maker's control. It is 
up to the group to identify all important and relevant actions and events in 
the particular problem domain. There are no restrictions on the number of 
decision or event nodes, nor is there any assumed standard length of time 
from one node to the next. | | 


Judgmental Values and Probabilities. The generation of a decision 


tree requires not only problem structuring, but also value assignment. A 
“value" is a judgmental estimate in numerical form as to the worth of a 
particular action or event outcome. This estimate can be made on a relative 
scale from 0 to 100, where 0 represents the "worst" possible situation and 
100 represents the "best" possible situation. Such an estimate is called 

a "utility" judgment. Each time additional action alternatives or event 
outcomes are added to the decision tree, a utility judgment is necessary 

for each of them. In addition, all event outcomes require an estimate of 
their likelihood of occurrence. This likelihood is given in the form of a 
probability estimate from 0% to 100%, where estimates near 0% mean that 

the event is highly unlikely to occur, and estimates near 100% mean that the 
event is very likely to occur. 
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Decision Rule. With the utility and probability information in hand, 
it is a simple mechanical procedure to determine which of the decision 
options at the root of the tree is the "best". Starting from the tip nodes 
and working backwards, every internal tree node can be assigned a value 
based on the values at the end of its branches. The value of each decision 
node is simply the maximum of the values on its branches. This rule reflects 
the notion that, given a choice, a rational decision maker will choose that 
option with the highest utility. The value of each event node, on the 
other hand, is the expected value of its branches. That is, on each branch, 
the probability estimate is multiplied by the corresponding utility and 
then summed with the values on the other branches. This rule reflects the 
fact that events are not under the direct control of the decision maker. 


When the nodal values are "rolled-back" to the root node, the initial 
branch with the highest value (expected utility) is recommended for adoption. 
Further, the final decision tree acts as a contingency plan for future 
decisions. Since the tree is time-oriented, the proper path can be traced 
as actual events occur. Every time a decision node is encountered, the 
branch with the highest expected utility should be followed. 


1.3 System Description 


The Perceptronics group decision aiding system is an integrated, 
computer-supported aid. Its purpose is to improve group decision making 
by permitting continuous participant interaction with the computer during 
the decision making process. The aiding methodology combines guided 
elicitation of a group decision tree with direct and immediate feedback of 
key elements in the tree-building process. The decision system aids the 
group process in five main ways: 
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1) Decision Tree Problem Structuring 
2) Full Group Participation 
3) Identification of Critical Issues 
4) Conflict Resolution 

) Decision Recommendations 


Each of these are discussed separately below. 


Decision Tree Structuring. Basically, the aiding system guides the 
group in the construction of the decision tree -- a structure that permits 
the formal representation of major decision alternatives as well as possible 
future consequences. Through use of the decision tree structure, the group 
members are able to focus their discussion on the issues that are most 
consequential in reaching a final problem solution. This structuring does 
not inhibit the formation of creative or innovative problem solutions to the 
decision problem. However, it prevents loss of valuable time in arguments 
about irrelevant issues. 


Full Group Participation. Each group member enjoys full and equal 
participation in the decision process through individual computer-entry of 
required probability and utility values. These inputs are aggregated with 
those of the other members to form a group value which is displayed to the 
group, and is open for discussion. By giving each participant an assured 
voice in the decision process, domination by one or two individuals is 
avoided. 


Identification of Critical Issues. As the decision situation is 
dissected, a more and more detailed analysis is required. Normally the group 
would be required to treat all aspects of the analysis equally, regardless 
of whether a particular area was worth the effort required to develop it. 
Through a technique called “sensitivity analysis", the aiding system 
automatically identifies those parts of the decision tree most critical to 
the final decision, and recommends these for discussion by the decision group. 
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In this way, the group 1s constantly led toward consideration of the most 
crucial issues, and toward a more efficient decision making process. 


Conflict Resolution. During the course of utility entry by 
individual group members, excessively large differences in value may occur 
on particular issues. The group decision system recognizes these value 
conflicts, and helps to resolve them. When a value conflict occurs, the area 
of conflict is identified, and a decomposition procedure based on the Multi- 
Attribute Utility Model (MAUM) is initiated. This procedure breaks the | 
area of conflict into its constituent attributes. The group then enters 
separate values for each of the attributes, with the objective of isolating 
the conflict to one or two. By finally resolving conflicts on a few detailed 
points, the group is generally able to come to an agreement on the overall 
area. 


Decision Recommendation. The Group Decision Aid recommends a course 
of action based, by decision analytical calculations, on the inputs contributed 
by all group members. Each alternative is given an expected utility value. 
The recommended alternative is the one with the highest utility. Values and ~ 
recommendations are available at every stage of the tree-building process. 
If a recommended alternative is clearly ahead of its neighbors, the group 
may elect to accept it at any time. If, on the other hand, expected utilities 
are tightly clustered, the group may elect to continue analysis until a 
clearly superior alternative appears, or until a decision is forced by a time 
deadline. 


A major advantage of the group decision aiding system is that there 
is nO minimum time necessary to obtain a final decision. The decision aiding 
process is structured into repetitive decision "cycles". Each cycle expands 
One aspect of the problem in greater detail, and a system recommendation on 
the best group decision is available after each cycle. Of course, the quality 
of the final decision will depend upon how much time is spent using the aid. 


However, preliminary experiments have shown that a typical group can complete 
40 to 50 decision cycles in about 4 hours. This level of detail is sufficient 
to provide all group members with a high level of confidence in the value of 
the final decision. 


An innovative design element in the group aiding system is the use of 
a skilled technician called the "intermediator". The primary function of the 
intermediator is to facilitate communication between the group and the computer. 
The intermediator both directs operation of the aiding program, and translates 
spoken requests from the group into computer language. This leaves the 
participants with only the simple task of entering numerical values at 
designated program points. As a result, virtually no participant training 
is needed to operate the system, and a group can begin work on its decision 
problem almost immediately after it is convened. 


1.4 Empirical Evaluation 


Empirical evaluation and testing of the group decision aid has played 
a major role in its development. Several experimental decision making sessions 
were included in the first-year effort. These experiments were performed by 
representative decision groups using a previously synthesized scenario. | The 
scenario, which involves the choice of reactions to an international terrorist 
event, was chosen to fulfill a number of important criteria. These include: 


(1) Credibility and interest to the military as well as to the test 
groups 


(2) Inclusion of options already familiar to the selected group 
participants 


(3) Reasonable complexity to allow a rich set of alternatives and 
~ events 


(4) Existence of significant judgmental issues which may form the 
basis for conflict in values 


Formulated by CACI Federal Systems Group, Washington, D.C. 
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Initial tests have shown a clear acceptance of the aid by the 
participants. It was found that a very small amount of time is necessary to 
become familiar with the system procedures even by people with no formal 
training in decision analysis or value estimation. Group members felt that 
they were helped significantly by the automatic guidance features of the 
computer program. At the same time, the participants felt that the 
intermediator's role was crucial. Presence of the intermediator provided the 
necessary level of confidence required to follow the prescribed procedures 
and use the aid efficiently. Thus the Group Decision Aid represents a unique 
and useful example of man-machine synergism. 


2. SYSTEM DESCRIPTION 
2.1 Decision Environment 


2.1.1 Conference Facilities. The Perceptronics Group Decision Aiding 
System occupies two adjacent rooms, with the computer support equipment 
separated from the decision room. The group decision room, shown in 

Figure 2-1, is presently designed for three participants and an intermediator, 
all seated at a conference table. The conference table is equipped with 
specialized data entry terminals for the participants as well as a 
keyboard-display terminal for the intermediator. Facilities also exist 

for both audio and video recording of group decision sessions. 


Built into the front of the conference table is an Advent color 
video projector. Information concerned with the tree elicitation process 
is projected onto the large screen video display, located at the front of 
the room. The display is capable of showing lists of discussion items, bar 
charts, and attribute graphs, as well as partial and full decision trees 
which accentuate critical paths. | 


Each participant has an individual data entry terminal on the 
table in front of him. The terminal, shown in Figure oo. has the 
capability to accept numerical and YES/NO entries. A large 8-digit LED 
display shows the current numerical entry, as well as the previous entry 
for comparison. At the left of the keyboard are indicator lights that prompt 
the participants to enter specific types of data at the proper time. These 
indicators are: | 


(1) ENTER VALUE 
(2) ENTER PROBABILITY 
(3) ENTER LEVEL 
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FIGURE 2-1]. A DECISION GROUP AND INTERMEDIATOR AT THE INSTRUMENTED CONFERENCE TABLE. 
THE ADVENT VIDEO PROJECTOR IS IN THE FOREGROUND. 
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FIGURE 2-2. A PARTICIPANT TERMINAL, SHOWING THE DESIGNATOR SIGNAL 
| ENTER CHOICE) AND LED FEEDBACK DISPLAY (3 
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(4) ENTER CHOICE 
(5) VOTE 
(6) ERROR | 
| (7) PRESS ( CLEAR 
The: data eenmndals are also capable of dispiavine auxiliary numerical ee 


information generated by the computer. This capability allows, for _ 
| example, previously entered values to be re-displayed at a later time as 


a reminder of earlier estimates. 


$2, 1.2 ~The Intermediator. ‘the intermedi ator ' S terminal, shown in 

Figure 2-3, contains a full alpha- numeric keyboard and a smal video 

display on which he receives program information essential to his functions 
and different from that shown on the large-screen display. The intermediator 


acts as the communications link between the ‘group and the decision aiding 


system. Action alternatives and event outcomes that are generated by the 

group are entered into the group decision system by the intermediator, 

using the alpha-numeric keyboard. He also leads the group through the 
decision process using guidance messages appearing on his small-screen 


_ display and interpreting, when necessary, information appearing on the 


large screen display. - The fol lowing enumerates some Lor the more important 
| functions of the ‘intermediator. 


(1) Defines the value and probability scales to the group members. 


(2) Enters names and/or abbreviations for decision alternatives 
and event outcomes as they are generated. 


(3) Activates various data displays for the group when requested. 


(4) Clarifies specific actions the group is to take at various 
points in the decision process. 


(5) Performs editing functions on entered data. 
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FIGURE 2-3. THE INTERMEDIATOR AT HIS TERMINAL, SHOWING THE FULL ALPHANUMERIC KEYBOARD 
AND PRIVATE COMPUTER FEEDBACK DISPLAY. 
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(6) Activates group conflict resolution procedure when required. 
(7) Alerts group when numerical data entry is required. 


(8) Requests hard copy of current decision tree as required. 


2.2 Aiding Processes 


2.2.1 Decision Cycle. The decision aiding procedure is organized into 
decision "cycles". A complete cycle represents the expansion of 

One node of the decision tree, and involves five major steps (See Figure 
2-4): | 


(1) Node Selection 
(2) Alternative and Outcome Generation 
(3) Value Elicitation 

(4) Conflict Resolution 


(5) Decision Analysis 


The first step in the cycle, node selection, is based on a system 
recommendation of the currently most critical node. Once the area of © | 
discussion has been established, the major action alternatives or event : 
outcomes , as the case may be, must be generated. Each alternative or 
outcome is then assigned a utility value and, for outcomes, a probability | 
value. If conflicts arise, the multi- attribute utility model is activated 
to resolve them. The current decision tree is then analyzed to determine 
the best decisions thus far, and to plan for the next decision cycle. 
This cycle is repeated for each node of the tree that is expanded. 

A recommendation for the best decision is available at the end of each — 
cycle, and the group is free to terminate the session or continue. - 


Node 
Selection 


Alternative 


Decision and Outcome 


Analysis Generation 


Value 
Elicitation 


Conflict 
Resolution 


FIGURE 2-4. DECISION CYCLE 
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2.2.2 Attribute Selection. Before the decision cycles can begin, the 
group must prepare for discussion by establishing the relevant attributes 
of the particular decision problem. The "attributes" are the underlying 
issues that will ultimately form the basis for argument or agreement. 

For example, if the decision problem is whether or not to build a nuclear 
power plant,some relevant attributes might be: (1) effect on environment, 
(2) safety of nearby residents, (3) effect on local economy, (4) effect 
on energy requirements, etc. As the attributes are mentioned and discussed, 
the intermediator places them on the large screen display where they 

can be edited, and finally accepted for use during the decision making 
process. 


All attributes are currently considered to have the same importance. 
That is, their "weights" with respect to each other are equal. This 
assumption was shown to be acceptable by Newman, Seaver, and Edwards 
(1976) provided there are no negatively correlated attributes. It is thus 
up to the intermediator to insure that the attributes are stated in such 
a way that a change in one will not cause another to change in the 
opposite direction. It is planned to incorporate a provision for | 
differential weighting of attributes into a future version of the group 
system. Further, experimentation with the group system has shown that 
group members sometimes find one or more attributes irrelevant to the 
particular action or event under discussion, and thus wish to delete 
certain attributes and/or add new ones. Facilities for changing the 
attribute list at any time are also planned. 


2.2.3 Next Node Selection. The aiding system uses a sensitivity 
analysis algorithm to. determine the most critical tip node, and recommends 
it for expansion by displaying graphically the path from the root node 

to the recommended node. The most "certtical" ttp node ts defined to be 
the one that ts most ltkely to cause a change tn the currently best 
tnittal braneh at the deciston tree root. The reasoning is as follows: 
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as the tree expands, values are placed on newly-generated action 
alternatives and event outcomes. These new values affect the values of 
the initial decision options, located at the beginning of the tree. The 
initial decision options are considered to be the most important, since 
the first action to be taken in the problem solution must be selected from 
among them. Since the root branch with the highest value is chosen as 
the system's recommendation for the best decision option, it follows that 
the tip node with the best chance of changing the currently highest root 
branch is the one that should be expanded. This tip node is defined as 
the most "sensitive", and is the one that is recommended by the system 
for discussion. Details of the sensitivity algorithm itself are given 

in Appendix A. 


2.2.4 Alternative and Outcome Generation. When the group discusses the 
possible opportunities for decision actions and the possible events that 
may occur, the group members take into account the decision path as 
described by the sequence of nodes from the root to the current node being 
expanded. The discussion naturally leads to a determination of the 
preferred node type. At this point, an indicator light on the data entry 
terminals prompts each group member to vote on one of the following three 
choices: 


(1) Decision Node 
(2) Event Node 
(3) Terminal Node 


A vote for a decision node indicates the group's desire to consider 
possible action alternatives, while one for an event node shows a desire 
to list possible event outcomes. If the group chooses to terminate the 
path, the elicitation cycle is aborted and a new node is selected for 
expansion by the sensitivity analysis algorithm. A terminated path is 
dropped from consideration in subsequent cycles. 


Through group discussion, relevant action alternatives or possible | 
event outcomes (as the case may be) are generated and entered into the 
system by the intermediator. The list appears on the large screen display 
and may be changed or edited as the group wishes before final acceptance. 
The list of action alternatives should be mutually exclusive. This means 
that although a number of decision actions may be possible, only one 
from the list can be chosen. The intermediator guides the group in 
structuring the alternatives so they are mutually exclusive. Event 
outcomes must not only be mutually exclusive, but must also be exhaustive. 
That is, it should not be possible for (1) two outcomes to occur 
together or (2) an outcome to occur which is not in the list. If the 
lists of alternatives or outcomes are not mutually exclusive, calculation 
of the values of internal tree nodes from those at the tips is invalid. 
The intermediator aids the group in complying with these requirements. 


2.2.9 Value Elicitation. The group members now enter their estimates 

of the utility or "worth" of the generated action alternatives (or 

event outcomes). Each alternative (or outcome) is taken, one at a time, 
(see Figure 25) and an indicator light on the data entry terminals 
prompts the group members to enter a value within the established range 

of 0 to 100. Each estimate is independent and unrelated to the others. 
Each, however, takes into account (1) the decision path in the tree | 
leading to the situation, (2) the possible opportunities for future actions 
offered by the situation, and (3) the possible events, good or bad, which 
could occur as a result of the situation. Even though values for early 
actions should theoretically take into consideration all possible subsequent 
event outcomes, such values are actually "provisional", since they will 
most likely be refined at a later time by expansion of the action node 

into possible events. Holistic values for actions, or chains of actions, 
seem to be less reliable and accurate if no events have been established. 
Thus, the group is encouraged by the intermediator to expand events after 
actions. Individual values are averaged, in the absence of conflict, to 
achieve a group value. 


DECISION PATH: 


ACTION: ATTACK 


What events might occur? 


1. Lose hostages and Mandero City 

2. Lose hostages and save city 

3. Save hostages and lose city 

4. Save both ! 
| 


VALUE ASSIGNMENT 


0 100 

———E 

Worst Best 
Effect Effect 


Enter a value and press the SEND key. 


FIGURE 2-5. COMPUTER PRESENTATION OF UTILITY 
ESTIMATION FOR EVENT QUTCOMES 


Probability estimation is required for event outcomes. Each event 
outcome is considered, one at a time, and its likelihood of occurrence 
(see Figure 2-6) is estimated. An indicator light on the data entry 
terminals prompts the group members to enter a probability for each 
outcome. The probabilities are entered in the form of percentages from 
0% to 100%, where probabilities near 0% indicate an opinion that the 
outcome is unlikely to occur and probabilities near 100% mean that the 
particular outcome will very likely occur. Probability estimates, unlike 
utility estimates, are related in the sense that the sum of the entries 
must be 100% over all of the outcomes. After anonymous entry of 
probabilities, they are averaged and entered into the tree. It has been 
previously shown that averaging probabilities is the least biased 
aggregation method (Dalkey 1977), and that other methods do not give better 
estimates of group opinion (Winkler, Murphy, Katz, 1977). 


2.2.6 Conflict Resolution. At some point in the elicitation of utilities 
for alternatives or outcomes, it is possible that members of the group 

will differ significantly on the values assigned. At this juncture, a 
closer look at the utilities associated with the node must be made. By 
closer, we mean a shift from "gestalt" or "holistic" value assignment to 
multi-attribute utility measurements (MAUM). The reason for this shift is 
that arguments over the assignment of utility usually reflect genuine 
disagreements about values. Multi-attribute utility measurement (breaking 
a holistic evaluation down into its component parts) can indicate 
explicitly the underlying values of each participant, show where and how 
much they differ, and in the process, frequently reduce the expanse of 

such differences. The system determines if a value conflict exists by 
calculating how far each group member is from the average. If there is 
group agreement, no further processing is necessary. However, if a 
conflict is detected, the Multi-Attribute Utility Model (MAUM) procedure 

is initiated. This procedure allows the group to decompose the alternatives 
in conflict into the underlying attributes that were established at the 
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beginning of the session (see Figure 2-7). By assigning utility judgments 
on each attribute separately, with respect to the overall] alternative, it 
is hoped that the group can isolate their disagreement to one or two major 
issues and resolve the conflict by a detailed discussion. 


The process of assigning utilities to each of the attributes may 
bring the group into immediate agreement with a quick resolution of the 
conflict. However, it is possible for the source of the conflict to 
come from one or more particular attributes. Figure 2-8 shows a histogram 
displaying the values entered by the group for each attribute. Each group 
member is represented by a differently colored bar. The attributes in 
conflict are identified by the system (by red titling) and become the 
targets for group discussion. Figure 2-9 shows a display of values for 
seven attributes in graph form, with the utility scale given at the left 
for reference. Each group member is represented by a differently colored 
line. At the bottom of the display, the attributes in conflict are shown 
in red. 


After the group has had sufficient time to discuss the relevant 
issues surrounding each attribute in conflict, a procedural decision 
must be made. The group may: 


(1) Re-enter values for those attributes in conflict. 
(2) Re-enter values for all attributes. 


(3) Average existing values. 


The group may elect to re-enter values for one or more attributes, hoping 
to reach agreement (Figure 2-10). If there is still conflict, the MAUM 
procedure is repeated. Finally, the procedure is terminated when the 
group either reaches agreement, or decides to average the existing values 
and continue node expansion. | 
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The conflict resolution procedure is described in detail in 
Appendix B. It should be noted that conflict resolution occurs for 
utilities only. There is, at present, no provision for probability conflict 
resolution; inclusion of such a capability is planned for the future. 
In addition, an alternative approach to conflict resolution, which is much 
closer to the philosophy of sensitivity analysis, is currently under 
investigation. Using this approach, a "conflict" would only occur when 
the values of each participant, taken individually, would cause a 
different in preference for one of the initial decision branches. A more 
detailed explanation of this approach may also be found in Appendix B. 


2.2.7 Decision Analysis. At the end of each decision cycle, a number 
of different summary displays are available for analysis of the decision 
tree generated up to that point. The major display is a graphic 
representation of the full tree, showing node types and branches with 
abbreviated labels. Figure 2-11 shows a sample of this display. Decision 
nodes are shown as solid boxes, event nodes as circles, and terminal nodes 
as trtangles. Open-ended branches indicate a potential place for further 
node expansion. On the right side of the figure near the center, a 
particular branch has been highlighted in yellow. This branch has been 
selected by the system as the most sensitive part of the tree and is 
recommended for expansion by the group on the next decision cycle. 

The tree may also be displayed in outline form, where the structure is 
shown by indentation. Figure 2-12 shows a sample outline-form tree with 
the current internal utility value displayed to the left of each node. 
Action nodes are shown in red, event nodes are in yellow, and terminal 
nodes are in green. 


At this time, the system recommends the best decision thus far, 
based on all of the information input up to that point. The recommendation 
for the best decision is that root branch with the highest expected | 
utility. This recommendation can be displayed along with the others for 
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comparison. If the group feels that the values of one or more decision 
options are too close, they may wish to continue expanding nodes until 

a clean distinction is reached or until the problem has been satisfactorily 
analyzed. Thus, the program may be continued or terminated at the completion 
of any decision cycle. 


2.3 Computer Support 


2.3.1 System Development Concept. Development of the group decision 

aid was based On a separation of the computer support system from the 

group decision facility itself. With this dichotomy in mind the conceptual 
framework of a decision aiding system was developed essentially independent 
of the computer resources required to support it. The color display 
system and participant terminals were chosen to best fit the decision 
making environment. Figure 2-13 shows the facilities arrangement. 
Consequently, the software was developed in a modular fashion, where the 
interfaces to the terminals used in the group decision facility were 
well-defined and transparent to the remaining aiding system. The 
interfaces are easily modified to accommodate different terminals. This 
development methodology enhances the transferability of the group aiding 
system to other decision facilities. The following sections describe 

the hardware resources and the software components used in implementing 
the system at Perceptronics. . 


2.3.2 Hardware. The group decision aiding system is implemented on a 
DEC PDP 11/45 minicomputer with 64K words of memory. The operating 

system used is UNIX, a versatile timesharing and multi-tasking system 
favored by research-oriented users of the PDP 11 series of computer. 
Capabilities of UNIX include full timesharing of 2-40 users, multi-tasking 
(spawning of independent processes), a comprehensive file-management 
system, and numerous compilers and software packages, including C, a 
Pascal-based systems programming language (in which UNIX itself is 
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written). Support is provided by Bell Telephone Laboratories and by a 
large (100-150) user group. | | 


Terminals. The decision makers and the intermediator are seated 
at a conference table facing the Advent large screen display system (see 
Figure 2-1). Situated in front of each participant is an Interface 
Technology 732 data entry terminal (DET) which is used for entry of 
numeric values and voting (see Figure 2-2). The DET's have an eight-digit 
LED display, numeric keypad, function keys, and eight indicator panels 
that are under program control. 


The intermediator has an Informer D301 terminal equipped with a 
16 line by 32 character CRT, full alphanumeric keyboard, and function 
keys. The display unit of the terminal can be rotated to obtain the 
optimum viewing angle. Using the Informer terminal, the intermediator 
can enter and edit lists of alternatives, query and direct the system, 
and receive reports from the computer aiding system on group performance. 


Graphics Support. The Genisco Model GCT-3000 programmable graphics 
system generates a TV-compatible, 8-color, full-graphics display with a 
resolution of 512 x 512 raster units. The graphic system contains its 
own very high speed (150 ns cycle time) fully programmable microprocessor 
with 4K of static random access memory (RAM) for program storage plus 
direct memory access (DMA) to the PDP 11/45 main memory. Any graphics image 
of rectangles, circles, vectors, conics, polygons, or other shapes can 
be displayed in user selectable colors. These can further be overlaid or 
annotated with alphanumeric text in any display location. 


The large screen display is the Advent 1000A color TV projection 
system which uses three color projection tubes to form a bright picture 
on a highly-reflective 7-foot diagonal screen. In the Perceptronics 
system, the Advent electronics are modified to accept the 10 MHz bandwidth 


output of the Genisco graphics display system. The projector is inset 
into the conference table. 


2.3.3 Software. The group decision aiding software was implemented in 

the C language from a top down design which facilitates program modifications 
as new ideas are incorporated into the system. There are four major 

software components (see Figure 2-14) totalling 6000 lines of code, plus 

an additional 2600 lines of assembly code composing the Genisco graphics 
operating system. The major software components control the following: 


(1) The images produced on the Advent large screen display. 


(2) The internal decision tree structure and computational 
procedures that manipulate the tree. 


(3) The interactions between group members and data entry 
terminals. 


(4) The operation and display of the intermediators terminal. 


Graphics Software. Three levels of software control the Genisco 
color graphics system. The lowest level functions, composed of Genisco 
assmbly instructions, provide the control interface between PDP 11 
software and the graphics system. Another set of functions at the middle 
level provides X, Y raster positioning, color selection, and operation 
mode when displaying text or geometrical shapes. The top level procedures 
provide simple C program access to a formatted screen of n lines by m 
characters. The Genisco system is easily initialized to a display position 
by designating line numbers and/or character column positions. Other 
procedures at this level can then write text or draw vectors on the display. 


Participant Interface. Data entry and display procedures control 
the interactive information flow between the decision makers and the 
system. Participants are prompted by lighted message windows for the 


GRAPHIC 
TREE 
DISPLAY 


~ GRAPHIC 
SYSTEM 


TEXT 
DISPLAY 


INTERMEDIATOR 


USER PARTICIPANT 
CONTROL INTERFACE 


DATA 
DISPLAY 


| PARTICIPANTS INTERMEDIATOR 


INTERFACE | 
DECISION 


UTILITY AND 
PROBABILITY 
ASSESSMENT 


DECISION 


SUPPORT 
SYSTEM 


CONFLICT 
RESOLUTION 


FIGURE 2-14. MAJOR SOFTWARE SYSTEMS 


various inputs required by the system, e.g., utilities, probabilities, and 
vote response. Feedback control is maintained by these procedures and 
information is displayed on the individual participant terminals to effect 
ease of terminal operation. 


Intermediator Interface. The software interface to the intermediator 
terminal provides system and group supervisory control. The intermediator 
1S provided system processing information at all times. Option lists 
displayed on the terminal at various points in the elicitation and node 
expansion cycle provide immediate control over system operation. The 
intermediator also assumes prime responsibility for program pacing during 
the decision aiding session. 


Decision Support System. The decision tree processing software 
expands the representative tree structure, elicits utilities and 
probabilities from individuals, analyzes values for conflicts, invokes 
the multi-attribute procedures to resolve conflicts, graphically displays 
histograms and line graphs to distinguish areas of conflict, and displays 
the graphical form of the decision tree on the large screen display. 

Other tree processing functions include rollback calcualtions and sensitivity 
analysis. There are also procedures to determine best path, select the 

best node for expansion, aggregate utilities and probabilities, and update 
the decision analysis profile during the decision making session. 


The the group proceeds through the decision cycles, a complete 
record of the computer interaction is stored for later retrieval and 
analysis. This record is called the "audit trail" and includes: 


(1) The list of initial selected attributes as well as those 
rejected. 


(2) A record of every action alternative and event outcome 
considered. 


(3) A listing of each utility and probability value input 
by the group members at each stage. — 


(4) A complete record of all conflict situations showing initial 
and modified attribute assessments. 


(5) A complete listing of the decision tree after each 
decision cycle including all current internal node values 
and probabilities. 


A sample audit trail is given in Appendix C. 


3. TEST AND EVALUATION 
Sel Test Scenario 


In order to test the capabilities of the Group Decision Aid, a 
reasonably complex and realistic scenario was required for the observation 
of user interface with the system, assessment of time required for 
completion of a complex task, and determination of the nature and degree of 
intermediator instruction and prompting required for efficient utilization 
of system features. A fictitious, but plausible, crisis scenario based 
on counter-terrorist actions was developed by CACI Federal under subcontract 
to Perceptronics. The guiding criteria for scenario design were: 


(1) Credibility and interest to the military as well as to the 
test groups. 


(2) Inclusion of options already familiar to the selected group 
participants. 


(3) Reasonable complexity to allow a rich set of alternatives and 
events from which to choose. 


(4) Existence of significant judgmental issues which might form 
the basis of intra-group conflict. 


The resultant scenario is summarized below. 
Scenario Summary. A U.S. B-52 carrying two nuclear weapons is 


flying in the vicinity of Shamba, a third-world ally of the United States. 
Due to problems with its onboard electrical system, the plane is forced 
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to land at Shamba's capital city of Savin. U.S. technicians are expected 
to arrive in four hours to effect repairs. Two hours after the B-52's 
landing, Shamba's government is overthrown in a coup by leftist rebels. 

The plane and its payload are captured and its crew executed; the U.S. 
Embassy in Savin is occupied and its personnel taken as hostages. The rebels 
Subsequently issue a list of demands and an ultimatum. The U.S. has four 
hours to accede in principle to their demands and 15 days to carry them 
out, otherwise U.S. hostages will be killed and a bordering neutral 
city-state, Mandero City, will be bombed with the B-52's payload. Two U.S. 
aircraft carriers equipped with surface-to-air missiles, attack air wings, 
and marine assault forces have been deployed to positions twelve miles 
from Mandero City. Joint Chiefs of Staff estimates indicate that these 
forces are adequate to either free the hostages or capture the plane and 
bombs. To accomplish both missions, these forces must be substantially 
augmented. The transfer of reinforcements could be completed within 2-3 
days. 


3.2 Test Procedures 


Three informal tests have been conducted to date, each with a 
duration of approximately four hours. The primary objectives of these 
initial tests have been to observe user performance, to obtain participant 
- evaluations of the utility of the system, to identify any remaining 
technical bugs, and to examine and further refine system features. Each 
test group has included three subjects. Two groups have been made up of 
research scientists, and one of graduate students in international relations. 
At the beginning of the test session, briefing booklets are distributed 
to participants, including simulated situation reports, cable traffic, 
press reports, and agency briefing updates, all designed to familiarize 
Subjects with the scenario. The subjects have about 20 to 30 minutes — 


to review and discuss their scenario briefing materials. The information 
provided in these materials has generally proven adequate, as few requests 
have been made for clarifications or additional information. Following 
their review of scenario briefing materials, the subjects are given a 

brief introduction to the equipment, the role of the intermediator, and 

the elicitation procedures to be used in the session. Such concepts as 
utility and probability estimation, attributes, actions, and events are also 
discussed. 


At the conclusion of the session, subjects are asked to complete 
attitude survey questionnaires which elicit assessment of their satisfaction 
with the group decision-making process and the usefulness of the decision 
aid. Subjects are provided hard copies of the final decision tree to assist 
them in making their evaluations. An audit trail, detailing all participants 
and intermediator inputs to the system, is also available for subsequent 
analysis. To date, monitoring of participant reactions and responsiveness 
to system features has been carried out by the intermediator. In the future, 
video tape recording of sessions will be made to further aid in detailed 
analysis. 


Sa Results 


In general, all three experimental groups worked quickly and 
efficiently with the decision aid. In all three sessions, groups were able 
to complete a 4 to 5 level tree, including 30 to 50 separate nodes, within 
their four-hour time limit. A typical decision tree is shown in Figure 
3-1. Subjects required no background in formal decision analysis, nor 
familiarity with decision tree construction, to employ the aid with ease. 
Furthermore, participants exhibited no tendency to avoid complexity in the 
course of their decision task, indicating that system features sufficiently 
clarified and simplified decision procedures such that confusion was minimal. 
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All three groups demonstrated a marked ability to adapt to the 
system and to use its features to best advantage in terms of time. 
Realizing that the conflict resolution process was rather time-consuming, 
for example, subjects tended to deliberate more before entering values, 
invoking the MAUM procedure only when actual value conflicts emerged. 
Finally, it was observed that participant attitudes changed during the 
course of the decision processes. Actions preferred at the beginning of 
the session were abandoned following deliberation and examination of possible 
Subsequent events. Technically, subjects demonstrated no difficulty in 
interpreting system displays, nor in following the sequential logic of the 
program. The sense of "being lost" after substantial development of a 
complex tree was invariably overcome by a review of graphic and verbal 
displays of the group tree. Some forms of display were preferred over others, 
although not consistently across groups. Two groups preferred graphic 
representation of value conflict, for example, while the third relied 
primarily on histograms for resolution. In the course of initial runs, 
Subjects often expressed the desire to restructure their trees. In addition, 
occasional terminal entry errors or immediate “second thoughts" highlighed 
the necessity for a re-entry capability as well as structural modification 
options. Implementation of such options is currently in progress. 


Interactions with the intermediator were observed to occur smoothly. 
Experience has shown, however, that the intermediator must guard against 
participant attempts to include him in the decision-making process. There 
is often a fine distinction between resolving technical or procedural 
problems and arbitrating between substantive disagreements. For example, if 
three events are suggested by the group, one of which does not meet 
requirements of mutual exclusivity, caution must be exercised by the 
intermediator to ensure that no inference is made as to the relative worth 
of the suggested events. 
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3.4 Performance Evaluation Measures 


As pointed out previously, evaluation is an essential part of 
system development. Methodologies for evaluating the group decision aid 
must meet two sets of criteria. First, they must assess both the objective 
and subjective dimensions of group performance. Second, they must provide 
timely results, which can be used to influence decisions concerning the 
‘immediate development of the decision aiding system. Subjective measures 
are generally well defined, and not difficult to obtain. | 


The objective dimension of group performance, which concerns the 
quality of deliberations, is more difficult to measure. It involves 
analyzing such structural properties of group deliberations as: 


(1) The number of alternatives considered by each experimental 
group (differentiation), 


(2) The extent to which each alternative considered is developed 
(for example, the amount of time and consideration devoted to 
each brance or subbranch), and 


(3) The integration of alternatives within a complete tree structure 
(for example, the relative development of branches considered 
in the early and later portions of the experimental session). 


In addition to examining the quality of the process of deliberation and 
decision-making, some “quasi-objective" assessments can also be made of the 
content and correctness of the decision reached by the experimental groups. 
The term "objective" must be qualified in this usage because there is no 
Single decision "solution" for most realistic crisis management problems; 


decision-makers with different values and objectives can and will come to 
different decisions. In this respect crisis management problems differ 
from intelligence questions, where there is an objective standard (ground 
truth) against which estimates can be evaluated. However, by using expert 
and experienced personnel, a set of "good" solutions can be generated. 
Using these "solutions" as standards, the quality of the final decisions 
reached by experimental groups, as well as the quality of the factors that 
they consider while arriving at these decisions, can be evaluated. 


In summary, the overall objective of further formal tests and 
experiments will be to determine, by quantitative as well as qualitative 
measures, the capabilities of the group decision aid in assisting decision 
makers in solving real-world decision problems, and to determine the extent 
to which the aid facilitates the decision process. It appears that such 
determination might be facilitated by using a multi-attribute scheme, 
much like that employed in the aid itself, both for the elicitation of 
Subjective judgments and for the combination of subjective with objective 
measures. 
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4. CONCLUSIONS 


4.] Advantages 


The specific advantages of the Perceptronics computer-based decision 
aiding system can be summarized as follows: 


(1) Natural Group Format 

(2) Structured Discussion 

(3) Domain-Independent Application 

(4) Equalized Participation 

(5) Selective Problem Expansion 

(6) Graphic Resolution of Value Conflict 
(7) Continuous Decision Recommendation 


The decision tree, while a formal structure, is general enough to be applied 
to a wide variety of decision problems. Any decision problem which can be 
thought of in terms of actions and events can be accommodated. Through the 
incorporation of sensitivity analysis, potentially large combinations of 
interacting actions and events are reduced to a workable set of critical 
issues. Each group member has an opportunity to voice his opinion (through 
the input of values) on the importance of each major issue. When conflicts 
do arise, they are treated efficiently, allowing the group to concentrate 

on quickly arriving at an optimal course of action. 


4.2 system Improvements 


The initial system tests, as outlined in Chapter 3, were highly 
successful. The major development goals of the system were realized, in 
that experienced groups were able to work effectively with the aid toward 
the solution of a realistic and representative decision problem. At the 
Same time, these trials identified a number of system areas in which 
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existing features could be modified, or new features added, in order to 
improve overall system performance and power. A number of the most 
important areas of improvement are listed below: 


(1) New conflict resolution approach (described in Appendix B) 


(2) 


would provide a less arbitrary method of identifying conflicts 
while focusing on resolution when it is really necessary. 


Modifiable attribute list would allow the deletion of 
irrelevant attributes and the inclusion of new attributes 
based on the specific alternative or outcome in conflict. 


Weighting of attributes according to importance would provide 
a more accurate conflict resolution result. 


Identification of probability conflicts as well as utility 
conflicts might provide for the more accurate representation 


of final outcome values. 


Restructuring Procedures that would allow the group to modify 
previously defined paths of the tree would permit more 
flexibility in action and event expansion. 


| Consistency checks and data validation algorithms would provide 


assurance that the value estimates given by the group were as 
good as possible. 


Weighting of participant values before averaging would allow 
greater levels of expertise or authority to be reflected in 


. the decision-making process. 


4.3 Development Areas 


Consideration has also been given to more basic questions in the 


development of the group aid. Among the major development areas identified 


are the following. 


New Problem Domains. Experience with decision problems in other 
domains 1S necessary to determine the commonalities and differences with 
respect to problem representation in the form of decision trees. Problems 
in medicine, business, negotiation, education, law, etc., as well as 
tactical and strategic military planning may have idiosyncrasies which 
must be accommodated. For example, a problem in business decision making 
may benefit from a monetary value scale rather than a utility scale. 
Further trials with groups making actual decisions in realistic or real- 
life environments would help identify some of these peculiarities. 


Data Base Integration. It is clear that the overall decision 
making process would benefit if the group decision aid were integrated 
with a computerized data base related to the problem at hand. With such 
integration, the group would have access to pertinent information at each 
Stage of tree construction. For example, when considering the actions to 
take at a decision node, the group might survey actions taken previously | 
in similar situations; or when trying to decide on what events might occur 
as a result of an action, the group might review the current local state- 
of-the-world as a guide to likely occurrences. 


Several options exist for effecting an integration with existing 
or developmental data bases. The simplest, but least appealing, option 
is just to run separate programs -- group decision aid and data base 
access -- without any inter-system communication. This precludes any 
automatic searching of the data base tied to the developing decision tree, 
and complicates use of the system displays. A second option for existing 
data bases would be to link individual users, or the intermediator, with 
the data base procram and use Unix filter and pipe facilities to control 
communication and output between the two programs. With this alternative, 
the user would effectively switch among systems and system interfaces. 
Ideally, however, users Of the group aiding system could access the data 
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base individually through their terminals, or by consulting the | 
intermediator, as an integral part of the group aiding system. Access. 

to the auxiliary data base would thus be with a well-engineered interface, 

compatible with that of the group aid itself. s 


The last option, while most attractive operationally, might well 
require extensive revision of the current aiding program for each separate 
data base dealt with. A compromise approach is the establishment of 
general-purpose "hooks" from the aiding program to the outside world. 
These hooks would make it feasible to attach the aiding program to a 
variety of data bases without extensive revision, and with an acceptable 
interface structure. 


Individual Tree Elicitation. In the current system, all group 
members must be physically present in a single conference room to construct 
a group tree. This appears to be an overly severe restriction in today's 
world of decentralized command and management. Two ways to remove this 
restriction appear feasible: (1) group members meet at the same time, but 
at different places, and communicate through teleconferencing facilities; 
and (2) group members work individually, probably at different times and 
places, to create separate trees; these trees are then merged to form a 
group tree, which may then be worked on further by the group members, 
locally or remotely. 

The first concept poses few problems, and could be readily © 
implemented using available technology. The second, that of individual 
tree merging, offers considerably more challenge, but promises greater 
rewards. The primary advantage of this approach, aside from its self-paced 
nature, lies in the opportunity for incorporating more creative solutions 
by individuals working totally outside the group structure. The primary 
difficulty lies in devising a merging method that preserves the individual 
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qualities of the trees and at the same time results in a fair and useful 
composite. Preliminary analysis indicates that merging cannot be wholely 
automatic, but must include a human element. The following paragraphs 
describe a preliminary scheme for computer-assisted tree merging, carried 
out within the general configuration of the present aiding system. 


The elicitation of a decision tree from an individual could be 
performed using a simplified version of the group elicitation program. 
The procedures for sensitivity analysis and outcome calculation are 
applicable to an individual as well as a group. For comparability of trees 
among participants, however, some additional rules are necessary. These 
rules need not be overly restrictive, but should result in some communality 
of tree structuring, without impacting the individualized analysis of the 
decisions. It is envisioned that each decision maker will interact with 
a small graphics terminal. The terminal should have capabilities of tree 
presentation, keyboard input, error analysis, sensitivity analysis and 
probabilitiy and utility aggregation. Also, a hard-copy printout of the 
resultant tree would be useful. 


Distillation of a "fair" composite tree from the set of individual 
trees can best be accomplished by a single decision analyst working at 
an interactive graphics terminal. Two operations are required: 
(1) identification and merging of structural components of the trees; 
and (2) aggregation of individual parameter estimates. 


Structural differences among the individual trees will emerge in 
spite of adherence to tree development rules. These differences will 
involve node labeling, degree of branching, and composition of the 
attribute sets. Distillation of the composite tree will be facilitated 
through the use of guiding rules similar to the following: 
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(1) Merging of the tree should proceed form left to right. 


(3) 


Merging of the parent nodes is most important since the 
treatment of later nodes will be conditional on this resolution. 


Inclusion of actions in the composite tree will depend on their 
importance and representation in the individual trees. If 

a majority of the individuals include an action in their 
formulations and evaluate the action highly, the action should 
be included. If an action is not well represented and is of 
low import, it should be deleted. 


Actions that overlap or are otherwise related between trees 
may often be analyzed from the resulting events. Set theory 
constructs may be used to recombine branches to arrive at 
comparable action sets. Similarly, event sets may be 
manipulated according to subsequent branches. 


Once the composite tree structure is established, the calculation 


of event probabilities, outcome utilities, and attribute levels is. 


Straightforward. The following rules are suggested: 


(1) Probability Aggregation. If, for a given action, common 


(2) 


events are present across all participants, the probability 
estimates can be aggregated using simple rules. Dalkey (1977) 
notes that either the arithmetic or harmonic mean are suitable 
estimators for aggregating such judgments. For later dialogue, 
a confidence interval may be assigned to the aggregated value 
depending on the variance of the individual judgments. 


Utility Aggregation. Aggregation of the holistic utility 
judgments for actions or outcomes may be derived by averaging. 
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Huber and Delbecq (1972) state that the arithmetic mean is 

an efficient and unbiased estimator. Comparability is present 
Since the judgments are made according to an externally defined 
scale. 


(3) Attribute Levels. The attribute levels need to be examined 
only for those outcomes that are found to be pivotal, ij.e., 
in which the range of individual utilities is wide enough to 
favor competing actions. Comparison of the attributes will 
presumably entail rescaling, since the anchoring points will 
differ between participants. 


The procedures outlined in this preliminary specification for 
tree merging require both human and machine contribution. The human is 
expected to use his pattern recognition capabilities to recognize 
morphological relationships and identify common elements. The computer 
aids by displaying tree structures and by providing an interactive medium 
for reorganization of the structures. Computer aiding is also used for 
sensitivity analysis, rescaling, aggregations, and error tests. The man, 
finally, assumes responsibility for direction and evaluation of the effort. 
As machine capabilities improve and as the tree-merging process becomes 
more tightly defined, the machine responsibilities will increase. 


The composite tree produced by the above methodology may be used 
in a variety of ways. It may be (1) used directly as an expression of group 
policy, (2) returned to each individual for modification and/or parameter 
estimation, or (3) used as an initial structure when convening the group. 


Direct use is justified if substantial agreement was evident during 


the merging process, or if time and resources make further modifications 
impossible. If substantial agreement on decision structure is present but 
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large differences in parameter estimation exist, then the composite tree 
may be returned to each individual for re-estimation. This is somewhat 
analogous to the Delphi procedure for achieving group consensus. Such 
an individualized re-estimation process has the advantage of being lower 
in cost and complexity than full convening of the group. The final 
possibility, convening the group for actual dialog, may be necessary if | 
major differences in tree structure must be ironed out, or if irreconcilable 
conflicts in parameter estimation are still present. The composite tree 
then serves as a guide for the group dialog. 


In fact, a composite tree used in this way might well provide an 
advantageous starting point for the type of computer-aided group discussion 
described in the preceding chapters, regardless of whether large differences 
are present or absent. That is, a group working with a composite tree 
Structure could focus on refinement of options and exploration of judgmental 
values, using their time together to establish a tree of greater precision 
and acceptability. 


5. REFERENCES 


Dalkey, N.C. Group Decision Making. School of Engineering and Applied 
Science, University of California at Los Angeles, CA. Report UCLA-ENG-7749, 
July 1977. 


Gardiner, P.C. and Edwards, W. Public Values: Multi-Attribute Utility 
Management for Social Decision Making. In Schwartz and Kaplan (Eds. ) 

Human Judgment and Decision Processes: Formal and Mathematical Approaches. 
New York: Academic Press, Inc., 1975. 


Hays, M.L., O'Connor, M.F., and Peterson, C.R. An Application on Multi- 
Attribute Utility Theory: Design-to-Cost Evaluation of the U.S. Navy's 
Electronic Warfare System. Decisions and Designs, Inc. (McLean, Virginia), 
Technical Report DT/TR/5-3, Contracts N00014-75-C-0426, N00039-75-C-0084, 
NO0024-73-C-1060, ARPA, Code 455, Office of Naval Research, October 1975. 


Huber, G.P. and Delbecq, A. Guidelines for Combining the Judgments of 
Individual Members in Decision Conferences, Academy of Management Journal, 
June 1972, pp. 161-174. 


Kelly, C.W. and Stewart, R.R. The Decision Template Concept. Decisions and 
Designs, Inc., December 1, 1977. 


Leal, A. An Interactive Program for Conversational Elicitation of Decision 
Structures. Ph.D. Thesis to UCLA School of Engineering, Report No. 
UCLA-ENG-REP-7666, June 1976. 


Leal, A. and Pearl, J. An Interactive Program for Conversational Elicitation 


of Decision Structures. Proceedings of the IEEE Systems, Man, and Cybernetics, 
May 1977, Vol. SMC-7(5):368-376. 


Newman, J.R., Seaver, D.A., and Edwards, W. Unit Versus Differential 
Weighting Schemes for Decision Making: A Method of Study and Some 
Preliminary Results. Social Science Research Institute Technical 
Report 76-5. July 1977. 


Raiffa, H. Preferences for Multiattributed Alternatives. Rand Corporation 
(Santa Monica, CA) Memorandum RM-5868-DOT/RC, April 1969. 


Sheridan, T.B. Community Dialog Technology. Proceedings of the IEEE 
Special Issue on Social Systems Engineering, March 1975, Vol. 63(3) :463-475. 
Winkler, R.L., Murphy, A.H., and Katz, R.W. The Consensus of Subjective 
Probability Forecasts: One, Two, Three, ...Heads Better Than One? 


Proceedings of the Fifth Conference on Probability and Statistics, 
November 15-18, 1977, pp. 57-62. 


5-] 


APPENDIX A 
SENSITIVITY ANALYSIS 


Traditional decision analysis demands expansion of the "complete" 
decision tree before utilities are assigned to the tip nodes and probabilities 
are assigned to the internal event nodes. If, however, utility and 
probability values are assigned as the tree is being expanded, it is possible 
to perform a sensitivity analysis on the partially constructed tree in order 
to determine the most critical (sensitive) places (Leal, 1976). Once 
determined, the most critical tip node can then be recommended for expansion 
and a great deal of time is saved by not expanding those parts of the tree 
that are not crucial to the initial decision. 


Such an early value estimation follows the same philosophy as that 
of tip node value assessment. The value should reflect the situation as 
described by the path to the node as well as an account of possible 
future events and action opportunities. Every value estimate is placed 
on a current "tip" node of the tree. Later, as the tree is grown, the 
tip node will become an internal node due to its expansion. At this time, 
its former value is replaced by the expected value of the subtree extending 
outward from it. Thus, initial value estimates are called "provisional" 
in the sense that later they will be replaced by more accurate estimates. 


Sensitivity analysis attempts to locate the most critical ("sensitive") 
node, that is, the node whose value must be changed the least in order to 
cause a drastic change in the current state of the tree. Figure A-I shows 
a simple tree which will provide an example for the description of the 
sensitivity analysis procedure. The root node has three main branches, 
labeled bl, b2, and b3. It is the objective of the decision analysis to 
choose one of these three alternatives. Assume that branch bl leads to a 
subtree (not shown) with a roll-back value of 5; branch b3 has a roll-back 
value of 2 from its sub-tree (not shown). Branch b2 leads to one event 
node with two outcomes: A and B. It has been previously estimated by the 


4 REQUIRED SENSITIVITY 
A 15 10 
B é, 2.9 


FIGURE A-1. BASIC SENSITIVITY DIFFERENTIAL 
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group that outcome A has a provisional value of 5 with a .2 probability of 
occurrence. Similarly, outcome B has a provisional value of 2.5 with a 
probability of occurrence of .8. With these values, the event node has an 
expected value of (5 x .2) + (2.5 x .8) = 3. Thus, bl is the currently best 
decision with the highest value of 5. 


Suppose that tip node A is chosen for future expansion. It's 
provisional value of 5 would then be replaced by a new value that comes 
from an aggregation of the new expanded tip nodes. If the newly replaced 
value at A is high enough, branch b2 could overtake branch bl and become 
the leading candidate for final choice. How high must the value of A be 
pushed before bl] goes from 3 to 5 (the currently highest initial value)? 
Node A must be raised from 5 to 15 -- an increase of 10. It may be said, 
then, that the "sensitivity differential" of node A is 10 since its value 
must be raised by 10 in order to cause a "decision switch" at the root 
node. If the sensitivity differential is calculated for all tip nodes in 
the tree, then the node with the lowest differential is the most sensitive, 
and it should be recommended for expansion. 


The formula for calculating the sensitivity differential can be 
expressed in terms of a recursive function that begins at the root node 
and computes the sensitivity for each node out to the tips. Thus, 
sensitivity calculations are carried out in a forward direction as contrasted 
with expected utility which is computed backwards from the tip nodes toward 
the root. The computation formula is: | 


S(n) for event nodes 
P(n) 


S(n) + V(n) - V(r(n)) for decision nodes 
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where S(n) is the sensitivity of node n, r(n) is the successor to n, P(n) 
is the probability along the branch from n to I(n) and V(n) is the current 
provisional value of n. The above procedure will, of course, produce a 
sensitivity differential of zero for all nodes connected to the highest 
initial branch. Thus, these nodes are compared with the second highest 
initial branch to determine how much they must be lowered in order to 
produce a decision switch at the root. 
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APPENDIX B 
CONFLICT RESOLUTION 


Figure B-1 illustrates a typical conflict situation. Three 
alternatives have been identified at a node. On two of these, the three 
group members are in good agreement with respect to overall value, and a 
mean value can be easily assigned. On the third alternative, the differences 
are much greater, and the conflict must be resolved in another way. 


The first task, then, is for the group decision aiding system to 
determine if a conflict actually exists. The current algorithm permits 
the participants to be within 10 value points (utility units) of the group 
mean (on the average) without causing a conflict. Conflict occurs when, 


_ ™M 
2X 
for: X = an 
M —_ 
1 ay 
a eae 10 


on a scale of 0 to 100 where x. 1s the value input by the ith participant 
and M is the total number of participants. This means that a conflict 
1S caused on an alternative when the average deviation from the group mean 
is above 10. When this situation occurs, the MAUM procedure is invoked 
for each alternative conflict. 


The basic idea of multi-attribute utility measurement is quite 
familiar (see, for example, Raifa, 1969). Every action has a value on a 
number of different dimensions or "attributes". The technique is to 
discover those values, one dimension at a time, and then to aggregate them 
across dimensions using linear average. Averaging utilities does the least 
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Group Member Utilities Group Utilities 


1 2 3 
Alternative 1 40 50 — 40 43.3. 
Alternative 2 70 60 80 70 
Alternative 3 2 25 78 « Conflict 


FIGURE B-1. TYPICAL CONFLICT SITUATION 
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"damage" to individual utility estimates according to Huber and Delbecq 
(1972). Although Gardiner (1975) and Dalkey (1977) suggest a weighted 
average aggregation of participant values, the weighting would depend on 
participant expertise. However, in the absence of differences in expertise, 
a straight averaging is recommended. 


Each group member is asked to consider the alternative under 
discussion and the level of performance of this alternative on each 
identified importance dimension. Then each group member, individually, 
assigns a utility value between zero and one hundred (the range of the 
utility scale), dimension by dimension, reflecting the utility of the 
alternatives performance on each dimension. At the conclusion of this 
step we will have a "decomposed" version of Figure B-1 such as shown in 
Figure B-2 for one particular alternative. 


The overall branch utility for each group member is calculated by 
using a Simple average: 


where Ui is the utility of the ith attribute dimension for the jth group 
member. Any conflicts in attribute levels can then be displayed as shown 
in Figures 2-8 and 2-9. If conflicts are not resolved, the group may elect 
to re-enter the MAUM procedure, or simply average the existing value and 
proceed with further tree expansion. 


An alternative approach to conflict identification and resolution 
is currently being explored. This approach adopts the philosophy taken 
by the existing sensitivity analysis algorithm which searches for the most 
"critical" tip node (i.e., the one that has the most chance of causing a 
decision switch at the root of the tree), while avoiding the somewhat 
arbitrary rule based on average deviation from the group mean. 
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Importance 


Dimension 2 3 

"attribute" — Utilit Utilit Utilit 
Alternative ] 50 90 95 
in 2 , 75 05 05 
Conflict 3 | 20 50 40 
4 65 50 55 


FIGURE B-2. DECOMPOSED UTILITY ESTIMATES 
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Consider an example of a decision node (Figure B-3) that has just 
been expanded. It is assumed, because this node was just expanded, that 
the sensitivity algorithm has chosen it as the most likely to cause a 
decision switch at the root node. In fact, the information regarding 
exactly how much the node's value must be altered in order to cause such 
a switch is stored in the computer memory at the node itself in the form 
of the sensitivity. If the values of each group member were taken separately 
(as though he were the only decision maker) it could be determined which 
member's values would actually cause a decision switch. If no one causes 
a switch, the group "agrees" in the sense that they all prefer the current 
initial alternative. Similarly, if all members cause a decision switch, 
they again "agree" since they all prefer a different initial alternative. 
However, if some cause a switch and others do not they "disagree" and the 
MAUM procedure can be invoked. 


This approach is much closer to the overall aim of tree construction 
which is to arrive at an agreed initial course of action for the particular 
problem at hand. When a conflict does arise, the MAUM procedure must then 
be applied to all alternatives simultaneously. There will no longer be an 
opportunity to select individual alternatives in conflict. This disadvantage 
is offset, however, by the advantage that the MAUM procedure will only be 
invoked when absolutely necessary. Furthermore, it will be easier to assign 
attribute values when considering all available alternatives rather than one 
or two in isolation, since each attribute can be held constant while comparing 
across alternatives. 
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Estimated Utility Values 


Alternative 1] Member | Member 2 Member 3 
43 62 50 

Alternative 2 22 30 21 

Alternative 3 5 50 80 


FIGURE B-3. EXAMPLE DECISION NODE 
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APPENDIX C 
SAMPLE AUDIT TRAIL 


The following computer output is a sample of the audit trail. 
It is taken from one of the preliminary experiments with the Group 
Decision Aid using the terrorist scenario. Each decision cycle is shown, 
along with utility and probability values entered by each of the three 


group members. 
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Suegsested Attribute List 


EFFECT ON INTERNATIONAL RELATIONS 
EFFECT OW DOMESTIC RELATIONS 
EFFECT ON MILITARY FOSITION 
EFFECT ON DIRECT EXPENDITURE 
EFFECT ON ECUNOMIC FOSITION 
EFFECT ON FRESTIGE 

EFFECT ON AORALITY 


Attributes Comsidered 


SAFETY OF HOSTAGES 

SAFETY OF MANDIIERO CITY 
RANGE OF B-S2 | 

U.S. MILITARY CREDIBILITY 
U.S. INTERNATIONAL FRESTIGE 
NOMESTIC OPINION 

FOSSIBLE NUCLEAR THREAT 
IMFACT ON SHAMBA 


1. 
a 
3. 
4. 


Fimal Attribute List 


SAFETY OF HOSTAGES 

SAFETY OF MANTERO CITY 
RANGE OF EB-S2 

U.S. MILITARY CREDIBILITY 
U.S. INTERNATIONAL FRESTIGE 
DOMESTIC OF INION 

FOSSIRLE NUCLEAR THREAT 
IMPACT ON SHAMBA 


Initisl Actions 


ATTACK 

NEGOTIATE / DELAY 
ACCETE TO DEMANDS 
TAKE TO U.N. 


DECISION TREE LESCRIFTION 


Node value rrecedes the node identifier 
Node rrobatbility follows mode descrirtion 


ACTIONS Es EVENTS Ti TERMINALS 


O A: ATTACK. O 

O AS NEGOTIATE 7/7 DELAY. O 
O As ALCEDE TO DEMANDS, oO 
O As TAKE TQ U.N... OO 


Noge Selected for Expansion 


"ATTACK? 


Alternetive Actions 


ExXfERimenr #2 


Bsttil 
Btt2 
BLS 
att4 
gstts 
Betts 
ste? 
stts 


NewU 


sttl 
stt2 
atts 
stt4 
atts 
BtLtSG 
Bstt7 
BtL8 


New - 


FREE HOSTAGES 
LESTROY AIRPLANE _ 
CAPTURE PLANE ANI! BOMES 


CARRIER FROTECTION OF M.C, 


DISPLAY STRENGTH 
Initial Utility Vselues 


F’ 
690 


3 
10 


35 


1 


9) 
0 


0 


F2 


70 


wQ- 


100 
70 
40 


FS 
80 


68S 


eo 
9S 


ae 


adam 
11 
14 


Boo 
21 


18 


Method of Resolution. 


MAUM 
MAUN 
MAUM 
MAUM 


conflict 


mee OS Re pt 


Resolve Conflict om Alternative 
"FREE HOSTAGES * 


AttrioOute Vslues 


F' 


1 


vO 


20 


i, 
deo 


w 


63 
9O 
6a 
1s 


JQ 


48 


Resolve Conflict on 
"DESTROY AIRPLANE" 


Attribute Values 


F 


1 


10 
100 
100 

70 

73 

79 

65 


vO 


79 


60 


0 
20 
100 
60 
69 
wO 


40 


mel? 


a ba 


F'3 


48 


FS 


49 


conflict 


CO Or 0 Oe 


Alternative 


conflict 


ooc orFCcr So 


a 


Resolve Conflict on Alternative 


"CARRIER 


Attribute Values 


FROTECTION OF M.C." 


Fi F2 FS adm conflict 


—Bttl a8) ZO JO ~- 13 1 
abt2 70 100 100 13 1 
gtt3 SO 30 100 27 1 
astt4 2 7O 100 27 1 
Betts 40 63 100 ee | J 
BELG 20 yO 100 2°? 1 
Btt7 10 2O 40 1i 1 
stts 70 70 80 7 0 
New 42? 5S 84 16 1 


Method of Attribute Conflict Resolution 
"Re-evaluste aettriboutes in Conflict" 


Attrioute Values 


Fi Fi2 FS adm conflict 
atti uO ra 6) uO 0 0 
att2 80 90 90 4 ¢) 
stts 70 79 70 0 0 
att4 40 a] @) 40 4 @) 
Betts 40 50 ae) 7 0 
3ttSG 40 uO 30 4 @) 
Bstt7 40 AO a0 4 0 
8stt8 70 ?0 80 7 @) 
Newt 4 62 50 3 0 


Resolve Comflict on Alternative 
“DISPLAY STRENGTH" 


Attribute Values 


Fl F2 FS acim comflict 
Betti 10 30 1) 1i 1 
stt2 20 50 rah @) 16 1 
st3 20 20. 20 0 0 
stt4 50 73 1060 14 1 
BtLLS uO 50 50 4 0 
Bstt6 60 uO 50 4 0 
stt7 30 20 0 11 1 
Btt8 Oo 10 0 4 0 
NewU Si 4i 36 3 8) 


DECISION TREE DESCRIF TION | 
Node value rrecedes the mode identifier 
Node rrobability follows node descrirtion 

A? ACTIONS E3 EVENTS Ti TERMINALS 


23 AS ATTACK. O 
we AS FREE HOSTAGES. 0O 
YS AS TESTROY AIRFLANE. 0 
93 At CAPTURE FLANE AND BOMBS. O 
22 Ae CARRIER FROTECTION OF M,C... 9 
36 As DISPLAY STRENGTH, 0 : 
O As NEGOTIATE 7 DELAY. 0 


O 
0 


Node Selected for Expansion 
"NEGOTIATE / DELAY" 


ettil 
BLL2 
Bstt3 
att4 
stts 
BELG 
Batt? 
atts 


NewU 


A} ACCEDE TO DEMANDS. 
At TAKE TO U.N.. 


filtermative Actions 
REAL TERMS 


DELAY FOR REINFORCEMENTS 
NELAY FOR WORLD 


Initial Utility Velues 


Fd 
10 
30 


3O 


Fa 
SO 
90 
20 


FS 
0 
100 
100 


OP INTON 


adm 
33 
29 
2°? 


Method of Resolution 


MAUM 
MAUM 
MAUM 


Resolve Conflict on 
"REAL TERMS" 


Attrioute Values 


Fd 
90 
90 
40 
10 
10 
10 
40 
43 


42 


2 
90O 
70 
70 
wO 
JO 
wO 
70 
76 


73 


F3 
100 


~100 


100 
JO 
vO 
40 

100 

100 


BO 


conflict 
1 
1 
1 


Alternative 


conflict 


eee eee OO 


1 


Method of Attribute Conflict Resolution 
"Re-evaluste attributes in Conflict’ 


ett 
BttL2 
Batts 
BtLt4 
BLES 
Batts 
awtt? 


stts 


Newll 


Attrinute Values 


FS 
100 
100 

90 

50 

40 

wO 

79 
100 


72 


9} 
Oo 


bObOUN HS DS 


or 


conflict 


ceoocoorcso 


ae 


Resolve Comflict or Alternative 
"TELAY FOR REINFORCEMENTS" 


Attrifhbute Values. 


Fd FS ES acim conflict 
BpetL 50 79 a0 7 0 
wL EL? 450 30 =O 4 0 
Btt3 40 vO B80 16 1 
BttL4 20 4O 40 4 0 
setts 35 30 40 3 8) 
atts 30 20 50 11 1 
stt7 30 50 70 7 0 
atts 80 SO BO 0 0) 


NewU 47 pa @) pals: 4 0 


Resolve Conflict om Alternative 
"TELAY FOR WORLD OF INION" 


Attribute Values 


Fd F'2 FS adm conflict 
stti 7a 10 uO 2 i 
Bett2 50 50 50 4 0 
att3 30 40 40 4 0 
wett4 20 10 0 7 0 
Btrts 30 10 0 1i 1 
BtLtS 30 10 30 9 0 
Btt?7 3O 20 33 10 1 
Betts 60 0 vO 24 1 
New 44 19 32 8 0 


DECISION TREE DESCRIFTION 
Node value rrecedes the node identifier 
_ Node rrobebility follows node descrirtion 
A: ACTIONS E? EVENTS T3 TERMINALS 


98 A? ATTACK. O 
ve A$ FREE HOSTAGES. 0O 
28 A’ DESTROY AIRPLANE. O 
96 As’ CAPTURE FLANE AND BOMBS. O 
29 A’ CARRIER FROTECTION OF M.C.. O 
36 As DISPLAY STRENGTH. 0 

67 Ai NEGOTIATE 7 DELAY. 0 
67 A’ REAL TERMS. 0 
ve As DELAY FOR REINFORCEMENTS. 0 
32 Ai TELAY FOR WORLD OFINION. 0O 

ACCEDE TO DEMANDS. 90 
TAKE TO U.N... O 


Node Selected for Expansion 
"ACCEDE TO DEMANDS * 


DECISION TREE DESCRIFTION 


Node vslue precedes the mode identifier 
Hode Frobspility follows nmoge descrirtion 


A+ ACTIONS Es EVENTS 3 TERMINALS 


98 AL ATTACK. oO 


57 


OTs 
O A: 


a2 As} FREE HOSTAGES, O 

“6 As TESTROY AIRPLANE. O 

93 At CAPTURE FLANE ANI BOMBS. O 

27 As’ CARRIER FROTECTION OF M.C.. O 
365 At DISPLAY STRENGTH. 0O 


A+ NEGOTIATE / DELAY. 9O 


67 Ai REAL TERMS. O 

ve As’ DELAY FOR REINFORCEMENTS. 0 
32 Ai TELAY FOR WORLT OPINION. O 
ACCEDE TO DEMANIS. oO 

TAKE TQ UsNe. O 


Node Selected for Exrsnsiori 


abt 
Batt2 
Btts 
Bstt4 
Betts 
we LG 
Btt?7 


"TARE TO U.N," 


Estimsted Frotsbilities 


Fy Po PS 


70 50 70 
30 35 29 
O 5 5 
fltermative Events 
CARRY OUT THREATS 
MODIFY DENANDS : 
THREATS NOT CARRIED OUT 
Initial Utility Veslues 
Pio OFS OBZ adm conflict 
@) 0 0 e) ) 
73 BO 100 10 | 
509 95 100 17 0 
Method of Resolution 
hAUM 


Resolve Comflict on Alternstive 


“MODIFY DEMANTS® 


Attribute Velues 


Fd 2 PS adm conflict 
90 SO 109 7 
rae, FO 100 4 
70 60 100 164 
40 50 30 4 
60 7u 80 8 
uO 7o BO 12 
60 To B80 B 


OKROOHOO 


stts 80 70 B80 4 0) 


NewW 68 #74 84 | é 0 


TECISION TREE TESCRIFTION 

Node value frrecedes the node identifier 

Node rrobsbhility follows node description 
A: ACTIONS EE: EVENTS T3 TERMINALS 


98 As ATTACK. O 
v2 A’ FREE HOSTAGES. 0 
J8 As DESTROY AIRPLANE. O 
98 A; CAPTURE PLANE ANDI BOMES, OO 
3? As’ CARRIER PROTECTION OF H.C... 90 
356 AS BPISFLAY STRENGTH. O 
67 At NEGOTIATE 7/7 DELAY. oO 
57 AS REAL TERMS. O 
ve As TELAY FOR REINFORCEMENTS. 0 
32 At DELAY FOR WORLD OFINION. 0 
; ACCEDE TO DEMANES. Oo | 
; TAKE TO U.N... 9 
O E? CARRY OUT THREATS. 66 
73 Es’ MODIFY DEMANDS. 30 
oo Es THREATS NOT CARRIED OUT, 3 


Node Selected for Exransion 
*TELAY FOR REINFORCEMENTS" 


Estimated Frobsabilities 


Fi F2 FS 
Bliti 65 40 4S 
g1lt2 20 30 30 
alt3 15 30 LU 


Alternative Events 
1. CARRY OUT THREATS 
2, SUCCESSFUL OPERATION 
3. MODIFY DEMANIS 


Initial Utility Values 


Fi FP? FS adm conflict 
slti ) 0 ) ) 0 
alt2 100 100 100 ) 0 
Bs1lts3 uO 60 uO 4 0 


TECISION TREE DESCRIFTION 
Noge value rrecedes the mode identifier 
Node Frohaebility follows node descrirtion 
A: ACTIONS E: EVENTS T? TERMINALS 


93 As ATTACK. O 


FREE HOSTAGES. 0O 
TESTROY AIRFLANE. oO 
CAF TURE PLANE ANDI BOMBS. oO 
CARRIER FROTECTIGN OF N.C... 0 
DISFLAY STRENGTH. 90 
6/7 4&3 NEGOTIATE 7/7 TELAY., 0 
67 AS REAL TERMS. 9O | 
338 Ai TELAY FOR REINFORCEMENTS. O 
O E3 CARRY OUT THREATS. SO 
1600 E$ SUCCESSFUL OFERATION. 26 
23 BE MODIFY DEMANDS, 23 
32 Ai TELAY FOR WORLTI OFINION, ©O 
T? ACCENE TO DEMANDS. oO 
A} TAKE TO U.N... 0 
O EE: CARRY OUT THREATS. 66 
75 Et MODIFY DEMANDS, 30 
SS FE? THREATS NOT CARRIED OUT. & 


“O 
8) 
D> D> YD D 


Node Selected for Exransion 
"REAL TERMS" 


Estimated Frobsbilities 
Fi FP? FS 
Balti 33 33 33 
alt? 33 33 33 
B1ItS 33 33 33 
Alternative Events 
1. CARRY OUT THREATS 
ar LO NOT CARRY OUT THREATS 
3, WMODIFY DEMANDS 


Initial Utility Values 


Pi. Fe PS adm conflict 
Bplti 0 oO 100 44 1 
@21t2 79 8O 100 ii 1 
B1ILS 40 Zo uO 13 1 


Method of Resolution 


tL. MAUM 
me MAUM 
3+ MAUM 


Resolve Conflict om Alternative 
"CARRY OUT THREATS" 


Attribute Velues 


Fil 2 PS acm conflict 
stt1 ¢) 0 0 0 ) 
Bstte O- 0 0 8) 0 
BLS 20 2O 20 0 0 
wsbtT4 0- 6) 0 0 0) 
wetts ®) O 8) 0 0 


stts 


0 8) O 0 @) 

Bett? 20 O 0 9 0 
Betts 8) 0 0 0 0 
2 = | 1 0) 


NewU y 


Resolve Conflict on Alternstive 
"TO NOT CARRY OUT THREATS" 


Attribute Values 


, Fi P2 FS adm conflict 
stti 70 20 80 7 0 
Btt2 60 BO 100 13 1 
Btt3 ra) 80 100 18 1 
sttr4 60 60 100 18 1 
Btts 50 FO 100 16 1 
BtELSG a 8) 80 100 18 1 
Bwtt7 40 90 100 14 1 
stts BO 100 100 9 @) 
New 61 34 98 13 1 


Method of Attribute Conflict Resolution 
"Re-evseluste sttributes in Conflict’ 


Attribute Vslues 


Fil F2 FS acm conflict 
etti 70 90 80 7 ) 
Btt2 S58) BO 100 9 0 
stts JO vO a) 0 ¢] 
Bett4 50 30 60 4 0 
atts 709 Zo BO 3 0 
BtLG 60 63 90 i2 1 
stt7 65 80 90 > ¢) 
w3tt8 BO 100 100 9 — QO 
New 655 7A Si a O° 


Resolve Conflict on Alternative 
"MODIFY PEMNANTS ° 


Attribute Values 


Fi P2 FS acm conflict 
Btt1 50 6g WO & 0 
BstLt2 460 6a JO ray 0 
Bstts pa @) uO uO 0 O 
Btt4 30 40 40 4 0 
BttS 40 AO 40 0 0 
sttG 40 30 6a 9 O 
sett? 50 19) 50 4 0 
sttLS 80 635 uO 10 1 
NewU ae a od 1 ') 


NECISION TREE TDESCRIFTION 
Node value rrecedes the mode identifier 


Node rrobebility follows node description 
A’ ACTIONS Es; EVENTS 73 TERMINALS 


93 As ATTACK. O 
we Ai FREE HOSTAGES. 9 
YO A’ DESTROY AIRPLANE. O 
78 A} CAPTURE FLANE ANI BOMBS. O 
uP AS 


CARRIER FROTECTION OF M.C.. O 
36 As DISFLAY STRENGTH. 0O | 
43 Ai NEGOTIATE / DELAY. 0 
43 A: REAL TERMS. O | 
3 E$ CARRY OUT THREATS, 33 
74 ES TO NOT CARRY OUT THREATS, 33 
ve ES MODIFY DEMANDS. 33 
38 As DELAY FOR REINFORCEMENTS. 0 
O Es} CARRY OUT THREATS. S90 
100 E$ SUCCESSFUL OPERATION, 26 
25S ES MODIFY DEMANDS. 23 
32 A$ DELAY FOR WORLD OPINION. 90 
> ACCELDE TO DEMANDS. 0 
> TAKE TO U.N.» 0 
O E? CARRY OUT THREATS. 466 
7o ES MODIFY DEMANDS. 30 
eo E3 THREATS NOT CARRIED OUT. 3&3 


Node Selected for Expansion 
"TELAY FOR WORLD OFINION®’ 


CARRY OUT THREATS 
NOT CARRY OUT THREATS 


Estimated Probabilities 
Fi Fi? PS 
alti 0 80 460 
alt2 40 20 40 
Alternative Events 


1. CARRY OUT THREATS 
2, NOT CARRY OUT THREATS 


Imitial Utility Values 


Fd F2 F's adm conflict 
alti 6) 0 0 0) '¢] 
alt? B80 100 100 > 0 


DECISION TREE TESCRIFTION 

Node value rrecedes the mode identifier 
Node rroostilitey follows mode descrirtion 
At ACTIONS E3 EVENTS T3 TERMINALS 


98 AS ATTACK. O 
we As’ FREE HOSTAGES. O | 
w8 As DESTROY AIRFLANE. O 


98 A’ CAPTURE FLANE ANI! BOMBS. 0 
S59 At CARRIER PROTECTION OF M.C.. 0 
34 At DISPLAY STRENGTH, 0 
! NEGOTIATE / DELAY. 0 
43 At REAL TERMS. 0 
3 Et CARRY OUT THREATS. 33 
74 Et IO NOT CARRY OUT THREATS. 33 
52 Et MODIFY DEMANDS, 33 
38 Ai DELAY FOR REINFORCEMENTS. 0 
0 Et CARRY OUT THREATS. 50 
100 Ei SUCCESSFUL OPERATION. 26 
S53 Et MONIFY DEMANDS, 23 
31 At DELAY FOR WORLD! OPINION. 0 
O Ei CARRY OUT THREATS. 66 
93 Et NOT CARRY OUT THREATS. 33 
O T! ACCEDE TO DIEMANIIS. 0 
25 At TAKE TO U.N... 0 
O Et CARRY OUT THREATS. 66 
75 Ei MODIFY DEMANDS. 30 
85 E? THREATS NOT CARRIEL OUT. 3 


Node Selected for Exrarsion. 
"CARRIER FROTECTION OF M.C.° 


SUCCEED IN ROMBING A.C, 
SHOOT DOWN B-S2 | 
NO ATTACK ON H.C. 


Estimsted Frohbabilities 


Ft Fi? FS 
sltl 20 2 50 
alt2 60 42 43 
BltS 20 . 30 2u 


Alternative Events 
1. SUCCEED IN BOMBING W.C. 
2. SHOOT DOWN B-5S2 
3. NO ATTACK ON A.C. 


Initial Utility Values 


Ft Fi? FS edm conflict | 
wlti 0 0 0 0 9) 
slt2 65 73 80 é @) 


s1lti vO JO 30 0 O 


DECISION TREE RESCRIFPTION 
Node veslue rrecedes the monde identifier 
Node rrobebility follows mode descrirtion 
A: ACTIGNS E: EVENTS T3 TERMINALS 


98 As ATTACK. O 
ve As FREE HOSTAGES. 9O 
78 As TESTROY AIRFLANE. O 
98 &3 CAPTURE FLANE ANI! BOMBS. 0 


49 At CARRIER FROTECTION OF M.C.. 0 
O E} SUCCEED -IN BOMBING 4.C., 25 
73 E? SHOOT DOWN K-52, 50 
YO ES NO ATTACK ON 4#.C., 25 
36 AS DISFLAY STRENGTH. 0 
43 As; NEGOTIATE / TIELAY. 0 
43 As REAL TERMS. 0 
3 Es’ CARRY OUT THREATS. 33 
74 Es [TO NOT CARRY OUT THREATS. 33 
ve ES MODIFY DEMANDS. 33 
38 A: DELAY FOR REINFORCEMENTS. 0 
O Es CARRY OUT THREATS. 30 
100 E: SUCCESSFUL OFERATION. 26 
23 E3 MODIFY DEMANDS, 23 
S31 As’ DELAY FOR WORLD OFINION. 0O 
O E3’ CARRY OUT THREATS. 66 
93 Es NOT CARRY OUT THREATS. 33 
>; ACCEDE TO DEMANDS. O 
; TAKE TO U.N... 0 
O E: CARRY OUT THREATS. 66 
7o ES MODIFY DEMANDS. 30 
85 E} THREATS NOT CARRIED OUT. 3 


Node Selected for Emrarnsion: 
"CAPTURE FPLANE ANI! BOMBS * 


ATTACK SUCCEEDS 
ATTACK FAILS 
FARTIAL SUCCESS 


Estimated Frotabilities 


Fd Po FS 
ait AS 70 65 
aglt2 a w 0 
B1t3 vO 20 33 


Alternmetive Events 


1. ATTACK SUCCEEDS 
2e ATTACK FAILS 
3. PARTIAL SUCCESS 


Initial Utility Values 


Fi P2 FS edm conflict 
alti 100 100 100 0 0 
slt2 20 0 e) 9 0 
slt3 40 oO 30 4 0 


NECISION TREE DPESCRIFTION 
Noge value fFrecedes the mode identifier 
Node rrobaebility follows mode descrirtion 
AS ACTIONS E3 EVENTS TT? TERMINALS 


77 At ATTACK. oO 
52 At FREE HOSTAGES. 0 


9B AY DESTROY AIRFLANE. oO 
77 Asi CAPTURE FLANE AND BOMES,. 0 
100 E} ATTACK SUCCEEDS. 60 
7 Es: ATTACK FAILS, 3 
47 Es FARTIAL SUCCESS. 36 
49 A; CARRIER FROTECTION OF M.C.. O 
O &s SUCCEED IN ROMBING N.C... 25 
73 Es SHOOT DOWN B-S2. 390 
~O Es NO ATTACK ON MC... 25 
36 As DISPLAY STRENGTH. O 
43 As NEGOTIATE / DELAY. 0 
435 A: REAL TERMS. 0O 
3 E? CARRY OUT THREATS. 33 
74 Ei DG NOT CARRY OUT THREATS. 33 
we E? MODIFY DEMANDS. 33 
38 A’ DELAY FOR REINFORCEMENTS. 0 
O E$ CARRY OUT THREATS. 530 
100 E; SUCCESSFUL OFERATION., 26 
33 ES MOLIFY DEMANDS. 23 
31 A: DELAY FOR WORLD OPINION. OO 
O E$ CARRY OUT THREATS. 6&4 
93 EE: NOT CARRY OUT THREATS. 33 
>; ACCEDE TO DPEMANES. oO 
; TAKE TO U.N... 0 
O E} CARRY OUT THREATS. 66 
3 Es? MODIFY DEMANDS, 30 
2 E3’ THREATS NOT CARRIED OUT. 3 


Node Selected for Exransion 
"DESTROY AIRPLANE’ 


KROMES & PLANE DESTROYED 
FLANE DESTROYED 
ATTACK FAILS 


Estimated Frrobtaebilities 
Fi F 2 FS 
s1tl 40 50 30 


alt2 55 35 40 
alt3 5S 5 0 


Alternative Events 


1. BOMBS @ FLANE DESTROYED 
2+ FLANE DESTROYETD 
3+ ATTACK FAILS 


Initial Utility Values 


Fed 2 3 adm conflict 
s1tl 9S 9O 100 3 0 
2z1lt2 69 7S BO é 0 


alts 0 0 0 0 0 


DECISION TREE DESCRIFTION 
Node value frrecedes tne mode identifier 
Node rrobsbility follows node descrirtion 


At ACTIONS E? EVENTS Ti TERMINALS 


77 As ATTACK. O | 
ve A$ FREE HOSTAGES. O 
72 As DESTROY AIRPLANE. O 
9S Ei BOMBS & PLANE DESTROYED, 43 
73 E3 PLANE DESTROYED. 43 
O E} ATTACK FAILS. 3 
7? #&AS CAPTURE FPLANE ANDI BOMES, 0 
100 Ei ATTACK SUCCEEDS. 60 
7 E? ATTACK FAILS. 3 
47 Es: PARTIAL SUCCESS. 36 
49 A’ CARRIER FROTECTION OF H.C... 90 
O E3 SUCCEED IN BOMBING M.C.. 25 
73 Es SHOOT DOWN B-S2. 30 
30 E$ NO ATTACK ON MC... 25 
36 €&$} DISPLAY STRENGTH. O 
43 At NEGOTIATE / DELAY. 0O 
43 A’ REAL TERMS. 0 
3 Es’ CARRY OUT THREATS. 33 | 
74 E$ DO NOT CARRY OUT THREATS. 33 
v2 E3 MODIFY DEMANDS. 33 
38 Ai DELAY FOR REINFORCEMENTS. 0 
O EE} CARRY OUT THREATS. 390 
100 E: SUCCESSFUL OPERATION. 26 
3S ES MODIFY DEMANDS. 23 
Si A: DELAY FOR WORLD OFINION. 0 
O E$ CARRY OUT THREATS. 6&4 
73 —&: NOT CARRY OUT THREATS, 33 
T3 ACCEDE TO DEMANDS. 90 
A}; TAKE TO U.N... O 
O EE? CARRY OUT THREATS. 66 
7o Et MODIFY DEMANDS. 30 
85 E: THREATS NOT CARRIED OUT. 3 


Node Selected for Beane an 
"FREE HOSTAGES® 


SUCCESS 
FAILURE . 
PARTIAL FAILURE 


Estimated Frobsbilities 


Pi F2 FS 
slti 25 20 27 
alt2 20 15 13 
alt3 55 65 60 


Alternative Events 
1. SUCCESS 


FAILURE 
FARTIAL FAILURE 


Wis 
~* Sd 


Initial Utility Velues 


Fi FZ ES adm conflict. 


alti 100 100 100 0 0 
alt2 0 0 0 —  O 0 
alts 70 70 wO : ? 


DECISION TREE DESCRIF TION 
Node vslue Frecedes the mode identifier 
Node rFrobsbilits follows node descriFftion 
At ACTIONS E: EVENTS T? TERMINALS 


77 &s ATTACK, O 
62 Ai FREE HOSTAGES, 0 
100 E: SUCCESS. 24 
O ES FAILURE. 16 
63 Es} FARTIAL FAILURE. 60 
72 A’ TESTROY AIRPLANE, 0 
95 E3} BOMERS & PLANE DESTROYED. 43 
73 E3 FLANE DESTROYED. 43 
O E$ ATTACK FAILS. 3 
77 & 3 CAPTURE FLANE AND BOMBS. 0 
100 E3} ATTACK SUCCEEDS. 60 
7 Es ATTACK FAILS. 3 
47 E3 FARTIAL SUCCESS. 36 
49 A: CARRIER FROTECTION OF M.C.. 9 | 
O &: SUCCEED IN BOMBING M.C.,. 25 
73 E3 SHOOT DOWN B-S2, S90 
2O ES NO ATTACK ON MC... 2S 
36 Asi TISPLAY STRENGTH, 0 
43 A’ NEGOTIATE / DELAY. O 
43 A$ REAL TERMS. 0 
3 E3 CARRY OUT THREATS. 33 
74 Es [DO NOT CARRY OUT THREATS. 33 
ve E? MODIFY DEMANDS. 33 
58 A’ DELAY FOR REINFORCEMENTS. 0 
O E3 CARRY OUT THREATS. S50 
100 Es SUCCESSFUL OFERATION,. 26 
23S ES MODIFY DEMANDS. 23 
31 A: DELAY FOR WORLD OPINION. 0O 
O Es’ CARRY OUT THREATS. 46 
93 E3 NOT CARRY OUT THREATS. 33 
>; ACCEDE TO DEMANDS, O 
>; TAKE TO U.N. O 
O — 3 CARRY OUT THREATS. 64 
73 Es’ MODIFY DEMANDS. 30 
8S E$ THREATS NOT CARRIED OUT. 3 
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